It drives me nuts that I cannot copy something from a ssh terminal when I use vim. I have no clue what this visual mode is meant to do besides anoying my.
My solution:
echo "set mouse-=a" >> ~/.vimrc no more visual mode -> marking and copying text from vim by using the mouse works again.